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ABSTRACT 

The newly revised Elementary and Secondary Education Act 
(ESEA 2001) has sweeping implications for how states collect, analyze, and 
use data about school and system performance. Policymakers must take a hard 
look at the design and capacity of their states' data systems, and determine 
what changes will have to be made to meet the requirements of the new law. 
This policy brief provides a short primer on data use, covering the topics of 
disaggregated data, ways of comparing school and student performance, and 
measurement of adequate yearly progress (AYP) ; information about longitudinal 
data systems and data structures needed for such systems that can measure 
student enrollment and progress over time; challenges of longitudinal 
systems, including cost and ensuring student privacy and confidentiality; a 
look at the resources available under ESEA 2001 for states to improve their 
information systems; and key issues for policymakers to consider in assessing 
the adequacy of their states’ information systems. The policy brief concludes 
with a brief list of resources for further reading about state information 
systems. (RT) 
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State Information Systems 



By Ravay 
Snow-Renner and 
Marga Torrence 



This is the second 
in a series of ECS 
reports examining 
the impact of No 
Child Left Behind 
(NCLB), the newly 
revised Elementary 
and Secondary 
Education Act on 
state policy and 
policymaking. Other 
reports will focus on 
accountability, 
school choice, lead- 
ership, literacy and 
teaching quality. 



The newly revised Elementary and Secondary Education Act (ESEA) has sweeping impli- 
cations for how states collect, analyze and use data about school and system performance. 

ESEA 2001 signals changing expectations about the quality and kind of evidence that will 
count in showing whether schools and states are doing their job of educating all students 
to proficient levels. The laws provisions - requiring annual student testing in grades 3-8 
in mathematics and science, the reporting and use of disaggregated achievement data in 
calculating adequate yearly progress (AYP), and specific sanctions for schools and districts 
not making AYP - put pressure on states to have the most accurate information possible. 

Collecting information about education programs and student achievement is not new for 
states. All but nine states, for instance, collect student-level information about achieve- 
ment on state tests. Many can report broadly about how various groups of students are 
doing. But in general, state systems aren't yet able to put together different types of infor- 
mation - like enrollment and achievement - to look at the performance of individual 
students. This means that they can't identify students who have moved from district to 
district, or from school to school, and they can’t look at individual student learning over 
time. 

In the past 10 or 15 years, new and more sophisticated ways of measuring student and 
school outcomes have been developed, along with more advanced technology that allows 
data to be transferred and used more efficiently. But existing state data infrastructures 
haven’t necessarily changed to be in step with the technology. 

The new ESEA requirements - including a broader collection of information and a speedy 
turnaround of state assessment data to local districts - present an opportunity for states to 
improve the usability of their data systems. The law does not explicitly require states to 
have specific database structures, but it endorses databases that link students’ test scores, 
the length of time they’ve been enrolled in given schools and graduation records over 
time. 
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In the coming months, policymakers must take a hard look at the design and capacity of 
their states’ data systems, and determine what changes will have to be made to meet the 
requirements of ESEA 2001. This policy brief provides: 

• A short primer on data use, covering the topics of disaggregated data, ways of 
comparing school and student performance, and measurement of adequate yearly 
progress 

• Information about longitudinal data systems that can measure student enrollment and 
progress over time 

• A look at the resources available under ESEA 2001 for states to improve their informa- 
tion systems 






^ “CLB POLICY BRIEF 




Key issues for policymakers to consider in assessing the adequacy of their states’ 
information systems. 
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A Short Primer on Data Use 

Disaggregating Data 

ESEA 2001 requires states to disaggregate data - separating, comparing and reporting 
information according to various student groups. Disaggregating data allows educators 
and policymakers to: 

• Look at achievement patterns by different groups 

• Identify the achievement gaps between different groups 

• Identify and study schools that "beat the odds" in raising student achievement, com- 
pared to schools with similar students. 

The emphasis on disaggregated data is in line with the new law's priority of improving 
achievement for all children. Achievement gaps occur between different groups of stu- 
dents in different states and regions, and it is only by first identifying these gaps that pro- 
grams and policies can be designed to narrow these gaps. 

Under ESEA 2001, states will have to disaggregate student achievement by the 
following categories: 

• Major racial /ethnic subgroups 

• Economically disadvantaged students (socioeconomic status) 

• English language learners 

• Students with disabilities 

• Gender 



• Migrant status. 

Most states will have to update their data structures to meet all of these requirements. 
Florida is the only state that currently reports student achievement at the local level, dis- 
aggregated by all of these subgroups. Many states report enrollment data on certain sub- 
groups of students, but few collect or report achievement by subgroup, especially at the 
school and individual student levels. 



One of the more problematic aspects of the legislation is the socioeconomic status data. 
Typically, schools have collected these data on a schoolwide level, but have not reported 
them on a student basis in terms of performance. 
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States may collect and report more information than is required in the legislation, but 
these categories will be key starting points for state policymakers and database coordina- 
tors to examine in relation to ESEA requirements. Glynn Ligon, a database specialist with 
Education Software Publishing, a consulting firm, has written a summary of ESEA 
reporting requirements, available at http://www.educationadvisor.com/. 
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Comparing School or Student Performance 

States can currently compare school or student performance in three different ways: 




• Comparing the same grade over time (cohort comparisons). This method compares 
how students in selected grades perform in the current year with how students in that 
same grade performed in previous years. For instance, the reading performance of 4th- 
grade students in 2001 is compared with the reading performance of 4th-grade students 
in 2000, 1999 and 1998 to get an approximate trend analysis. (It is approximate because 
different students are in each of the 4th grades that are compared.) 




• Tracking grades over time (quasi-longitudinal comparisons). This approach also uses 
cohorts of students, but the comparisons are of the same general cohort of students over 
time, rather than different groups of students. This approach requires regular testing. A 
quasi-longitudinal comparison might compare the reading scores of 4th-grade students 
in 1998 with the 5th-grade reading scores of students in 1999 and the 6th- and 7th-grade 
reading scores of students in 2000. 




• Tracking individual students over time (longitudinal comparisons). A true longitudi- 
nal comparison tracks the progress of individual students - rather than groups of stu- 
dents - over time. This type of comparison also requires regular testing. A longitudinal 
comparison of students might compare each student's growth over a year with the 
state's average growth. These types of comparisons require databases that can track 
student enrollment and achievement over time. 




Measuring Adequate Yearly Progress (AYP) 

ESEA sets expectations for states, schools and districts to help all students in the state 
reach proficiency in 12 years. To know where schools and districts are in relation to this 
goal, states are to define and calculate adequate yearly progress. Some alternatives for 
states are to: 




• Choose performance standards and define AYP objectives in terms of increasing the per- 
centage of students who score at the proficient level or higher on each assessment. For 
example, a state may need to bring 60% of its students up to proficient or advanced lev- 
els in 12 years. A state choosing this approach might define an AYP objective as having 
5% more students (60% divided by 12 years) perform at proficient or advanced levels 
per year. 




• Define several performance levels (such as novice, basic, proficient and advanced) and 
set AYP targets based on a combination of changes in the proportion of students at each 
level. In this model, schools might receive partial credit for moving students from the 
novice to the basic level, full credit for moving students from novice or basic to profi- 
cient levels and a small bonus credit for increasing the percentage of students in the 
advanced category. 
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For more information 
on issues covered in 
this policy brief, 
please visit the ECS 
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Site at 
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ecsmain.asp?page=/ 
html/issues.asp. 
Select Accountability 
(K-12) and then 
select Data Storage 
and Use. 



• Assign scores to performance categories and judge improvement in terms of increased 
average scores. In this model, student improvement across all performance categories 
will lead to higher average scores in the system. 

Longitudinal Data Systems 

Longitudinal student-unit databases provide the best, most accurate information for both 
policy decisions and decisions at the district and school levels. They can provide informa- 
tion about student growth over time that can then be linked with the teachers, programs 
and schools that have served those students. They also can provide fairer comparisons of 
schools than data systems that rely on cohort comparisons, because they ensure school 
performance is based only on students who have been continuously enrolled in that 
school. Finally, because they can match student records over time, they offer a way to 
follow student progress statewide and to verify the accuracy of district information - 
particularly about student transfers and dropouts. 

Longitudinal data that can be disaggregated by different groups provides useful, accurate 
information about school and student progress. It allows educators and policymakers to: 

• Look at student growth by school for different types of students. For example, are some 
middle schools preparing Hispanic students for high school better than others? Are they 
providing those students with a greater "value-added" experience? 

• Identify which programs work best for which students at which ages 

• Identify the relationship between early achievement levels and later student success 




• Look more accurately at patterns of mobility and dropout rates across student groups. 

Longitudinal comparisons also have advantages over cohort comparisons - which use 
"snapshot" data. School ratings based on snapshot data can change radically from year to 
year - especially for schools in neighborhoods that are changing fast. In situations where 
successive 4th-grade classes look very different from year to year, these snapshots do not 
provide clear pictures of school performance. They don’t show where students came from, 
how long they've been in the school or how well they have done in previous years. 

Longitudinal data can also take student mobility into account. It is not fair to hold schools 
accountable for the test scores of students who only enrolled there the week before the 
test, because those schools have had little opportunity to make a difference with those stu- 
dents. It is not a fair measure of school productivity when test scores rise and fall from 
year to year based on differences in groups of children. Only comparisons that track stu- 
dent learning over time - and can take into account student mobility - can offer solutions 
to these problems. 
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Two Examples of Longitudinal Data Systems 

Tennessee’s Value-Added Assessment System (TVAAS) is an example of a longitudinal 
accountability model. Beginning in 1997-98, Tennessee started testing every student in 
grades 2 through 8. TVAAS examines year-to-year gains of each student and compares 
them with gains made by a normative sample for the same subject in the same grades. 

The data are used to look at the "value added" by a teacher or by a school, compared with 
the value of other teachers and schools. A legislative review of the TVAAS is available at: 
http:/ / www.comptroller.state.tn.us/ orea/ reports / tvaas.pdf. 

A second example is the reporting model developed by Just For the Kids (JFTK) of Austin, 
Texas. The JFTK model reports school results in reading and mathematics. It calculates 
school performance based only on the students who have been in that particular school 
for two years or more - which is made possible by a data structure that can track student 
enrollment and achievement over time. JFTK is working with several states to develop 
these data infrastructures. More information is available at http: / / www.just4kids.org/ . 

Data Structures Needed for Longitudinal Data 

Longitudinal comparisons emphasize the goal of improved learning for all students. 
Making such comparisons requires a comprehensive database that is capable of tracking 
student enrollment and achievement information over time and across school and district 
lines. The database has to be able to look at individual student information, not just infor- 
mation about groups of students. 

There are three key characteristics of a longitudinal student-level database that are needed 
for optimal accuracy: 

• Two collections of student-level data are conducted per year: (1) in the fall, student-level 
enrollment, socioeconomic and program participation data, disaggregated as required 
for ESEA (this is not a student head count by school, but rather a separate database with 
one record for each student) and (2) in the spring, test data for all students in tested 
grades. Why? This helps to satisfy ESEA requirements for accurate dropout rates and 
identifies students who have been enrolled in English as a Second Language (ESL) or 
bilingual programs for three years. It also ensures the accuracy of demographic and pro- 
gram participation information, provides a check of whether some districts are having 
students "disappear" from spring testing data and makes it possible to report separately 
on the proficiency of students who have been continuously enrolled for three years or 
longer. 

• A statewide student-identifier system is in place to help match student records over 
time and as students change schools and districts. Why? This helps to satisfy ESEA 
requirements for accurate dropout rates and correct identification of students who have 
been enrolled in ESL or bilingual programs for three years. It also can be used to create 
"value-added" measures of student growth and evaluate the long-term effects of 
programs and policies. 
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States with student 
longitudinal database systems 
(17) 


States developing student 
longitudinal database systems 
(10) 


States without student 
longitudinal database systems 
(23) 


Arkansas 


Alaska (2003) 


Alabama 


Connecticut 


Arizona (2003) 


California 


Delaware 


Ohio (2003) 


Idaho 


Florida 


Washington (2003) 


Illinois 


Louisiana 


Nevada (2004) 


Indiana 


Maryland 


New York (2004) 


Iowa 


Massachusetts 


Rhode Island (2004) 


Kansas 


Minnesota 


Colorado (No date set) 


Kentucky 


Tennessee 


New Jersey (No date set) 


Maine 


Texas 

Georgia (2002) 
Hawaii (2002) 
Michigan (2002) 
Mississippi (2002) 
Oregon (2002) 

South Dakota (2002) 
Vermont (2002) 


New Mexico (No date set) 


Missouri 

Montana 

Nebraska 

New Hampshire 

North Carolina 

North Dakota 

Oklahoma 

Pennsylvania 

South Carolina 

Utah 

Virginia 

West Virginia 

Wisconsin 

Wyoming 
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Note: States in the third column either have no plans to match student records statewide by identi- 
fier, no student-level enrollment data collected on a statewide basis, and/or no statewide student- 
achievement testing. 



ESEA Resources for Information Systems 

ESEA provides some resources to help states reconfigure their information systems. These 
include: 



• Under Title VI, Part A, Section 6111, the U.S. Department of Education may provide 
funding to help states that have already developed the required assessments in grades 
3-8 for the purpose of improving the dissemination of performance information or to 
assist in linking student achievement, enrollment and graduation records over time. 

• Title I, Part E, Section 1501(b) allows for federal guidance and technical assistance to 
states in developing and maintaining management information systems. 



• Title V, Section 5121, subsection (6) allows states to use funds made available under that 
section for "support for arrangements that provide for independent analysis to measure 
and report on school district achievement." 

Key Considerations for States 

In assessing the adequacy of their current information systems within the context of ESEA 
2001, state policymakers should consider these key factors: 
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• The relative costs and benefits of revising the present structure to a student-level, 
longitudinal structure. Costs may be fiscal - in terms of expanding current data 
structures and developing unique student identifiers - or they may be political - involv- 
ing privacy issues and the relative trustworthiness of snapshot vs. longitudinal data. 
Costs will vary from state to state, and should be weighed against the benefits of better, 
more useful information. 

• Whether to adopt a more centralized or decentralized data model. The most cost- 
effective response to ESEA is to adopt a centralized data model, in which the state 
serves as the central repository of district and school information. For some large states 
that already have a comprehensive centralized structure, like Florida and Texas, this is 
a viable strategy. An alternative option for states is to develop a technology-supported 
system where the state collects some information and districts keep other information. 
Data may flow up and down the system based on reporting needs. A decentralized 
approach helps to preserve local autonomy and data ownership. In this option, a chal- 
lenge is to ensure data quality and standardize collection and reporting formats 
statewide. 

• Whether cross-state collaborative database efforts are viable. Oregon, Idaho and 
Washington database administrators, for example, are working collaboratively to devel- 
op a unique student-identifier system across all three statesADregon has developed a 
method for assigning unique identifiers (piloted in fall 2001) that Washington and Idaho 
are also beginning to use. When this system is fully operational, it will be possible not 
only to track student transfers across districts, but also across states. 

• Ways in which education data can be made more actionable - not only at the state 
level, but also for local educators. A longitudinal database offers more reliable informa- 
tion than snapshot models, but more needs to be done to help administrators, parents 
and teachers use state accountability results to improve student achievement. Combina- 
tions of frequently collected local data, supplemented by annual state accountability 
results, show promise. Reporting results is not enough. Educators and parents need 
models and assistance in learning how to use performance data to improve practice. 



Resources 

A new ECS publication. No State Left Behind : The Challenges and Opportunities of ESEA 2001, 
summarizes the ESEA 2001 law, looks at where the states stand in regard to requirements 
of the new law and suggests policy questions to consider when deciding how to respond 
to ESEA. (GP-02-01, 70 pages, $12.50 plus postage and handling) 




Additional information about the benefits of a state longitudinal student-level data system 

is available in a 2001 Education Week article by Chrys Dougherty of JFTK. See 

http: / / www.educationweek.org / ew / ewstory.cfm?slug=33dougherty.h20&keywords= 

Chrys%20Dougherty. 
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National Center for Education Statistics (2000). Building an Automated Student Record 
System: A Step-by-Step Guide for Local and State Education Agencies. Retrieved March 20, 
2002, from the World Wide Web: 

http:/ / nces.ed.gov /pubsearch/pubsinfo.asp?pubid=2000324. -r n 
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